Adaptive Constrained Kinematic Control using Partial or Complete Task-Space Measurements

Recent advancements in constrained kinematic control make it an attractive strategy for controlling robots with arbitrary geometry in challenging tasks. Most current works assume that the robot kinematic model is precise enough for the task at hand. However, with increasing demands and safety requirements in robotic applications, there is a need for a controller that compensates online for kinematic inaccuracies. We propose an adaptive constrained kinematic control strategy based on quadratic programming, which uses partial or complete task-space measurements to compensate online for calibration errors. Our method is validated in experiments that show increased accuracy and safety compared to a state-of-the-art kinematic control strategy.Comment: Accepted on T-RO 2022, 16 Pages. Corrected a few typos and adjusted figure placemen

Vitreoretinal Surgical Robotic System with Autonomous Orbital Manipulation using Vector-Field Inequalities

Vitreoretinal surgery pertains to the treatment of delicate tissues on the fundus of the eye using thin instruments. Surgeons frequently rotate the eye during surgery, which is called orbital manipulation, to observe regions around the fundus without moving the patient. In this paper, we propose the autonomous orbital manipulation of the eye in robot-assisted vitreoretinal surgery with our tele-operated surgical system. In a simulation study, we preliminarily investigated the increase in the manipulability of our system using orbital manipulation. Furthermore, we demonstrated the feasibility of our method in experiments with a physical robot and a realistic eye model, showing an increase in the view-able area of the fundus when compared to a conventional technique. Source code and minimal example available at https://github.com/mmmarinho/icra2023_orbitalmanipulation.Comment: 7 pages, 7 figures, accepted on ICRA202

MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization

Surgical robots are controlled using a priori models based on robots' geometric parameters, which are calibrated before the surgical procedure. One of the challenges in using robots in real surgical settings is that parameters change over time, consequently deteriorating control accuracy. In this context, our group has been investigating online calibration strategies without added sensors. In one step toward that goal, we have developed an algorithm to estimate the pose of the instruments' shafts in endoscopic images. In this study, we build upon that earlier work and propose a new framework to more precisely estimate the pose of a rigid surgical instrument. Our strategy is based on a novel pose estimation model called MBAPose and the use of synthetic training data. Our experiments demonstrated an improvement of 21 % for translation error and 26 % for orientation error on synthetic test data with respect to our previous work. Results with real test data provide a baseline for further research.Comment: 8 pages, submitted to IROS202

Virtual Fixture Assistance for Suturing in Robot-Aided Pediatric Endoscopic Surgery

The limited workspace in pediatric endoscopic surgery makes surgical suturing one of the most difficult tasks. During suturing, surgeons have to prevent collisions between tools and also collisions with the surrounding tissues. Surgical robots have been shown to be effective in adult laparoscopy, but assistance for suturing in constrained workspaces has not been yet fully explored. In this letter, we propose guidance virtual fixtures to enhance the performance and the safety of suturing while generating the required task constraints using constrained optimization and Cartesian force feedback. We propose two guidance methods: looping virtual fixtures and a trajectory guidance cylinder, that are based on dynamic geometric elements. In simulations and experiments with a physical robot, we show that the proposed methods achieve a more precise and safer looping in robot-assisted pediatric endoscopy.Comment: Accepted on RA-L/ICRA 2020, 8 Pages. Fixed a few typo

Photography-based taxonomy is inadequate, unnecessary, and potentially harmful for biological sciences

The question whether taxonomic descriptions naming new animal species without type specimen(s) deposited in collections should be accepted for publication by scientific journals and allowed by the Code has already been discussed in Zootaxa (Dubois & Nemésio 2007; Donegan 2008, 2009; Nemésio 2009a–b; Dubois 2009; Gentile & Snell 2009; Minelli 2009; Cianferoni & Bartolozzi 2016; Amorim et al. 2016). This question was again raised in a letter supported by 35 signatories published in the journal Nature (Pape et al. 2016) on 15 September 2016. On 25 September 2016, the following rebuttal (strictly limited to 300 words as per the editorial rules of Nature) was submitted to Nature, which on 18 October 2016 refused to publish it. As we think this problem is a very important one for zoological taxonomy, this text is published here exactly as submitted to Nature, followed by the list of the 493 taxonomists and collection-based researchers who signed it in the short time span from 20 September to 6 October 2016

Robot-aided endoscope control under laparoscopic surgery constraints using dual quaternions

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Elétrica, 2014.Este trabalho é dividido em duas contribuições complementares acerca do uso de manipuladores robóticos seriais para auxílio em cirurgias laparoscópicas. Primeiramente, técnicas conhecidas para robustez a singularidades e utilização da redundância são adaptados para o uso dos quatérnios duais unitários, que têm algumas vantagens sobre as matrizes de transformação homogênea, enquanto não possuem as singularidades naturais de representações mínimas. A performance das técnicas adaptadas são avaliadas em uma simples tarefa simulada. Utilizando estas técnicas, podemos controlar um robô para auxílio em procedimentos laparoscópicos. Diferentemente de robôs cirurgicos especializados, um robô serial pode ser utilizado para diferentes procedimentos, diluindo os custos em vários procedimentos. Neste cenário, a segurança do ponto pivotante deve ser garantida por software. Esse trabalho apresenta uma nova estratégia de controle para um endoscópio usando manipuladores robóticos com ponto pivotante remoto programável. As referências para o movimento do endoscópio são geradas intuitivamente pelo cirurgião. O método é avaliado em um ambiente cirúrgico simulado e apresentou resultados satisfatórios em termos erros de posicionamento e geração do ponto pivotante. _______________________________________________________________________________________ ABSTRACTThis work is divided in two complementary contributions concerning the use of serial-link robotic manipulators in a laparoscopic surgery setting. At first, known techniques for singularity robustness and redundancy exploitation are adapted to the use of unit dual quaternions, which have some advantages over homogenous transformation matrices concerning compactness, while not having singularities natural to minimal representations. The performance of the adapted techniques is evaluated in a simple simulated task. Using those techniques, we can control a manipulator robot to aid in laparoscopic procedures. As opposed to specialized surgical robots, a serial robot might be used for different procedures, lowering the involved costs. In such scenario, the safety on the pivoting point must be assured by software. This work presents a novel control strategy for controlling laparoscopic tools attached to robotic manipulators that makes use of a programmable RCM. The tool movement references are generated intuitively by the surgeon. The method is evaluated in a simulated surgical environment and presented satisfactory results, in terms of pivoting point generation error and tool positioning error

DQ Robotics: A Library for Robot Modeling and Control

Dual quaternion algebra and its application to robotics have gained considerable interest in the last two decades. Dual quaternions have great geometric appeal and easily capture physical phenomena inside an algebraic framework that is useful for both robot modeling and control. Mathematical objects, such as points, lines, planes, infinite cylinders, spheres, coordinate systems, twists, and wrenches are all well defined as dual quaternions. Therefore, simple operators are used to represent those objects in different frames and operations such as inner products and cross products are used to extract useful geometric relationships between them. Nonetheless, the dual quaternion algebra is not widespread as it could be, mostly because efficient and easy-to-use computational tools are not abundant and usually are restricted to the particular algebra of quaternions. To bridge this gap between theory and implementation, this paper introduces DQ Robotics, a library for robot modeling and control using dual quaternion algebra that is easy to use and intuitive enough to be used for self-study and education while being computationally efficient for deployment on real applications.Comment: 12 pages, 10 figures, 5 tables. This version has slightly different content from the IEEE typeset version. Also, the title has changed from the first submitted version, which was "DQ Robotics: a Library for Robot Modeling and Control Using Dual Quaternion Algebra.

SmartArm: Integration and validation of a versatile surgical robotic system for constrained workspaces

Background: With the increasing presence of surgical robots minimally invasive surgery, there is a growing necessity of a versatile surgical system for deep and narrow workspaces. Methods: We developed a versatile system for constrained workspaces called SmartArm. It has two industrial-type robotic arms with flexible tools attached to its distal tip, with a total of nine active degrees-of-freedom. The system has a control algorithm based on constrained optimization that allows the safe generation of task constraints and intuitive teleoperation. Results: The SmartArm system is evaluated in a master-slave experiment in which a medically untrained user operates the robot to suture the dura mater membrane at the skull base of a realistic head phantom. Our results show that the user could accomplish the task proficiently, with speed and accuracy comparable to manual suturing by surgeons. Conclusions: We demonstrated the integration and validation of the SmartArm.</p

The effects of different levels of realism on the training of CNNs with only synthetic images for the semantic segmentation of robotic instruments in a head phantom

Purpose: The manual generation of training data for the semantic segmentation of medical images using deep neural networks is a time-consuming and error-prone task. In this paper, we investigate the effect of different levels of realism on the training of deep neural networks for semantic segmentation of robotic instruments. An interactive virtual-reality environment was developed to generate synthetic images for robot-aided endoscopic surgery. In contrast with earlier works, we use physically based rendering for increased realism. Methods: Using a virtual reality simulator that replicates our robotic setup, three synthetic image databases with an increasing level of realism were generated: flat, basic, and realistic (using the physically-based rendering). Each of those databases was used to train 20 instances of a UNet-based semantic-segmentation deep-learning model. The networks trained with only synthetic images were evaluated on the segmentation of 160 endoscopic images of a phantom. The networks were compared using the Dwass–Steel–Critchlow–Fligner nonparametric test. Results: Our results show that the levels of realism increased the mean intersection-over-union (mIoU) of the networks on endoscopic images of a phantom (p&lt; 0.01). The median mIoU values were 0.235 for the flat dataset, 0.458 for the basic, and 0.729 for the realistic. All the networks trained with synthetic images outperformed naive classifiers. Moreover, in an ablation study, we show that the mIoU of physically based rendering is superior to texture mapping (p&lt; 0.01) of the instrument (0.606), the background (0.685), and the background and instruments combined (0.672). Conclusions: Using physical-based rendering to generate synthetic images is an effective approach to improve the training of neural networks for the semantic segmentation of surgical instruments in endoscopic images. Our results show that this strategy can be an essential step in the broad applicability of deep neural networks in semantic segmentation tasks and help bridge the domain gap in machine learning.</p